Self-Adaptive Parallel Processing Neural Networks with irregular Nodal Processing Powers using Hierarchical Partitioning

The architecture and working of the Artificial Neural Networks are an inspiration from the human brain. The brain due to its highly parallel nature and immense computational powers still remains the motivation for researchers. A single system-single processor approach is highly unlikely way to model a neural network for large computational needs. Many approaches have been proposed that adopt a parallel implementation of ANNs. These methods do not consider the difference in processing powers of the constituting units and hence workload distribution among the nodes is not optimal. Human Brain not always has equal processing power among the neurons. A person having disability in some part of brain may be able to perform every task with reduced capabilities. Disabilities weaken the processing of some parts. This inspires us to make a self-adaptive system of ANN that would optimally distribute computation among the nodes. The self-adaptive nature of the algorithm makes it possible for the algorithm to taper dynamic changes in node performance. We used data, node and layer partitioning in a hierarchical manner in order to evolve the most optimal architecture comprising of the best features of these partitioning techniques. The adaptive hierarchical architecture enables performance optimization in whatever condition and problem the algorithm is used. The system was implemented and tested on 20 systems working in parallel. Besides, the computational speedup, the algorithm was able to monitor changes in performance and adapt accordingly.